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Abstract 

In this contribation, we propose two new descriptors based on complex Fourier desci^tors to characterize a 
shape and its defbtmatioii in time. The aim of the latter is to characterize a segmented moving object as 
more or less rigid. For this» we use a method based on conq^lex Fourier descrq)tors. 



1. Introduction 

Much semantic iiiforraatu>n may be extracted from die shape of an object and its defomiation in time. For 
example^ in a video-surveillance application, the rigidity of a moving region will allow to dififexentiate 
pedestrians fix>m vehicles. 

However, when vehicles are driving away &om the camera, the 2D shape changes due to the perspective 
effect. This may be locaO^ approximated as an affine trans&trmation. In order to cope with this possible 
variation of scale or translation, the descr^tor has to be invariant to basic geometrical transformations such 
as translation, rotation or scaling. Furthennore, we need a scalable descriptor to be able to describe the 
shape and its deformation with more or less precision. 

To answer this need, we developed a new shape descrQ>tor based on complex Fourier descr^itors, which is 
invariam fay translation, rotation and scaling. Then we extracted a compact shape deformation descriptor by 
measuring the variability of the difierent frequencies in time. 

2. Complex Fourier Descriptors: definition and properties 

2.1. Definition 

Complex Fourier descrq)tors consist in a lossless representation of a shape comour G. A contour is defined 
as a set of points surrounding a sur&ce. Depending on the sampling, points are not necessarily connex. The 
length of the contour is the number of points used to describe it and therefore depends on the sampling. 
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Complex Founer descriptors are an equivalent freqoencial descriptbn and not a parametric representatbn. 
They are defined by: 

Equation 2-1 

z =■ X -h rv 

" " stands for the coordinate of the point of G, written as a complex number. Real part is 
absciss and imaginary part is ordinate. 

L stands for the length of G and ^ for the number of fiequency bins. 

These descrqitors have the same meaning as in signal processing: 

• Low frequencies, for ^ arouml ^ and ~ ^ , give a coarse idea of the shape 

AT 

• Hi£^ frequencies, for ^ around ^ ) represent fine details. 

This means that if two contours are very similar but for small details or for a small bcal part, the first 
coefficients will be very close, whereas the last ones will be completeb^ different. Besides, if the shape is not 
rigid, the shape contour wiQ change and so do the first coefiBcients. Of course the last ones will change as 
weU, but will not be significant. Hence, first coef&:ients aim at clustering diape contours. Intrinsically, 
complex Fourkr deso^tors axe a scalable representation of the contour. 

• ° stands for the continuous component (DC or Direct Current) and repiesents the non-normalized 
centroid of the contour. 

7 

• 1 is the radius of the circle whose sur&ce is equivalent to that of the shape, which can be interpreted as a 
scale parameter. 

• ^* and ^^'^ I < k < N ^ similar but opposite properties. ^ is the immber of actions regularly 

, IL ^ 

\<k< — 

spaced on the unity circle. ^ , ^ repiesents the munber of tension actions on the unity circle 

N 

~-hl<k<N 

towards the outside, whereas ^ , ^ represents the number of pressure actions towards 

the inside. 

• The phase ^* of * locates the action on the circfe. 



2.Z Properties 

2,2. i.Pr^iminaries 

Suppose that a contour is translated by ^ , rotated by ^ and scaled by a ^tor ^ to obtain ^ such as 

r r 

I and 2 have the same immber of pomts. Then there exists a simple relation between thecompfex 

„ . . . z' o<k<N ^ o<k<N ^r, I 

Founer descriptors * * of ^ and * * of 2; j 
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2^ ^f-'t X&^{i^)Zj^, q<,k<N 
Equation 2-2 

Making invaiiant fay translation, rotation and scaling is equivalent to cancelling the effects of ^ ^ ^ and 



2,Z2.Translation invariance 



^ is a continuous component and is therefore contained in ^ 
Z 

By not considering ^ , the set of coefficsnts 



translation invariant. 



2,2,3.Rotation and starting point invariance 

By then considering Ae set of coeffic^nts ^ 1 ^ ^ < ^ } ^j^^j^ abs{ ) modulus 

of , ^ flAs(Z^ \ 1^^<^}^ phase invariant. As both the starting point and a rotation induce 

a move of the phase, that is a multiplication by , the descriptor is rotation and startmg point 

invariant. 

2.2.4.ScaIe factor invariance 

We are now focusing on: 

abs(Zj,^)=: Aabs{Zj,^} \<k<N 



By finals considering the set of coefficients 
also scale invariant. 



, the resulting descr^or is 



2 Z 
Unfortunate^, ^ is not known, but present in each * . It is chosen to normalize by one of the descr^tors. 



7 

Since ^ is known to be a scale &ctor, each 



{abs[^ \<k<N} 

e, V I y is ( 



\<k<N 



win be divided by 



abs{Z^ 



Hence, v w jg translation, rotation and scale invariant. 

2,2.S.Contour iength invariance 

Equation 2-1 was estabhshed for two contours of same number of points . If their number of points differ, 
then their fi^queocial descrqition will also defor; difiference of length can be interpreted as a difference of 
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sampling. To cancel the influence of length, one contour nuist be resanq>led to the length of the other. 

By choosing for ^ a power of two, we can take ^ ~ L ^ which makes the description also sampling 
invariant. As a matter of feet, if we downsample the contour A from ^ points to ^ 

< OTj ^ ^ jggj frequency bins of each descriptor will correspond exactly to the same 

frequency, because the frequency 1^ ^ remains the same. Convffsely for upsampling from ^ 
pointsto ^2=2-\W2>'Wi, 

2.2.6, Compaction property 

t *» 0 0 > will be a truncated hst of the complete list of the 

conqilex Fourier descrq>tor5 necessary to describe the shape losslessly. The resulting teconstnicted ^i^>e 

N 1<.N <N 

wiD be a filtered version of the initial shape. The number of descriptors to retain 

depends on die complexity of the contour. However, ^0% g}} coefQcients are necessaiy to obtain a well- 
reconstructed contour with very few artifacts. 

2.2.7. Robustness to incomplete view 

Experiments have shown that the Fourier descnptors are sometimes very similar, sometimes completely 
different. As it both depends on the contour and the percentage of occlusion, it is safe to say it is not robust 
to incomplete view. 

2.Z8.Scalabiiity 

As stated in 2.1, con:q>lex Fourier descr^ois are intrinsically scalable: the higher the frequency, the finest 
the descrq>tion. 



3. Shape descriptor 
3- f - Descriptor definition 

The input data is a binaiy mask of an object san^led on a regular grid. The object has no holes. It is not a 
fractal object either. Beforehand, the contour of the object nmst be extracted, then resampled for its number 

of points to be a power of two ^ ~ ^ , That way, we take"^ ~ ^2 m the FFT. 

3.2. Specifications of ttte proposed siiape descriptor 

The descriptor should not only contain the necessaiy information on the shape but also be a supreme 
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summary of the fiifl infonnation avaflable at start. 
We propose the ^flowing MPEG-7 descriptor. 

(C C ) 

• Centroid ^ ^ : coordinates of the centroid of the contour. 

• Angle ^ : angle between horizontal and mam axis of the contour. 

• Size of the origiaa] contour^ : size of the contour after rcsanq)lfng. 

• Set of ordered Fourier coefficients ^ : set of invariant Fourier coefficients. 

• Size of the Fourier coefficients set ^ : size of the preceding set, ^ < P ,P is necessarily odd. 

• Scale : scale parameter. 

Here is the corresponding C structure: 

typedef struct Shape Descriptor { 

/* Centroid */ 
long center x; 
long center y; 

/* Angle */ 
float theta; 

/* Size of the original contour, after resampling (N) */ 
long size of contour; 

/* Set of Fourier coefficients */ 
float * Fourier Coefficients; 

/* Size of the set of Fourier coefficients <P) */ 
long size Fourier Descriptors Set; 

}; 

3.3. Extraction 

These are the steps which lead to a set of invariant Fourier coefficients: 

• Compute the two eigenvectors and the two eigenvahies from die contour and store angle as the angle ^ 
between the eigenvector associated with the biggest eigenvalue and the horizontal ^ is known modulo 

• Compute the FFT on the resampled contour of size ^ . to obtain * 0 <k < N ^ 



Store centroid as: 



_ ite(Z,) 

Jfai(Zo) 
N 
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Take modulus of each 
Store ^^^^^ as: 



abs(Zi) 



scale = 



N 



Divide each 2^A:<iV^ }^ ^^^(^x ) and store as Fourier coeflBcients 1 < 7 < 

Depending on the ^plication, choose the fis^ number ^ out of ^ Fourier coefBcients to keep. 



Store Fourier coefi&^ients 



in the foQowing order. 



(I) 

2;...zj.«*{|},*,[^)...»fc 



(z 



abs 



3.4. Matching 



0 w 

Given two sets of Fourier descrq^tors ^ and ^ , we want to compare their similanty. We will not take 
into account the position nor the angle, which do not chafacterize the shape itself and can be treated 



2 , then we 



P P 

separately, if the two sets are of dtSerent sizes, ^ and ^ respective^, and for instance 
p 

must compare the first ^ Fourier coefficients of the two sets. 

* /* (lr\ 

Considering that for one set, values ®' ^ at each frequency bin of order ^ are of diSerent order of 
magnitude, it is relevant, for each frequency bin, to normalize the difference of values between the two sets 
by the magnitude at the current fiequency bin. To harmomze the difiEerence of magnitude between 

frequencies, it has been chosen to sum relative errors between corresponding frequency bin values *^®' ^ ^ 
of the two descrq)tors. 

IHnally, we should consider that the coarse structure (low frequencies) prevails over fine details (hi^ 

frequencies), and therefore introduce a weig^iting function which privileges low frequency range at 

the expense of high frequency range. Therefire, it sets the infiiience of details in the final result. 

^ win denote the dissimilarity fimction and ^ the corresponding similaiity function. Return values are 
between ^andl. 
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a(0 ,0J=-^ 
E(x,>/) = 



X 

y-x 



SI x>y 
si y>x 



k=l 

5ifOT = l-A(© ,02) 



4. Shape deformation descriptor 
4. f . Descriptor defini^on 

The input data is a segmented video sequence of an unique object, that is a sequence of binary masks. The 
shape descriptor of the contour at each frame wiU be computed stored for processing, as described in 3.3 

This descr^tor is based upon the sh^>e descriptor exposed above. 

• Normalized deviation of the scale : normalized deviation of the scale parameter over the video sequence. 



• Maximal size of the original contours 

N 



the maximal size of the original contour sszes^ over the 



video sequence. is an item of the shape descrq>tor. 

• Normalized deviations of each Fourier coefRdent : normalized deviations of each Fourier coefficient 
over the video sequence. 

* Size of the set of normalized deviations of each Fourier coefficient^ : size of the preceding set 



Here is the corresponding C stmctiu^: 



typedef struct ShapeDef ormationDescriptor { 

/* Normalized deviation of scale */ 
float Deviation of scale; 

/* Maximal size of the original contours in the video sequence (N max) 

*/ 

long Maximal Size of Original contours; 
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/♦ Nonnalized deviation on Fourier coeflScicnts */ 
float *Deviaton of Fourier Coefficients; 

/* Size of the set of normalized deviations of Fourier coefficients * / 
long Size of Fourier Coefficients Set; 

}; 



4.2, Extraction 

Deviation of the scale factor and of each Fourbr coefiBcienl over the video sequence is calculated by using 
the standard deviation. Dividing by the n^an provides a normalization of the deviation. 

The size of the set of Fourier coefficients may vaiy along the video sequence, but as the £%quency lap 

remains the same, ^ Fourier coefficient ^ of ' fiume will be averaged with the ^ Fourier coefficient 

^ of frame. 



• Calculate the mean of ^^^l^ over the video sequence, 

• Calculate the mean of each Fourier coefficient ^ over the video sequence, 

• Calculate the standard deviation of ^^^^^ over the video sequence, 

Z' 

• Calculate the standard deviation of each Fourier coefi&:ient ^ over the video sequence, 

• Divkie die standard deviation of ^^^^ by its mean and store as , 

Z' ^ 

• Divide each ^ by its mean and store as ^ 

4.3. Matching 

Although a matching function is not relevant for our shape deformation descrqstor, because sh^ 
defbrmadon descrq)tors are not intended to be compared, we provide one anyway. The following function 

quamifies the similarity between two shape deformation descriptors and ^ . 

The number of normalized deviations of Fourier coefficients impHed in the calculation depends on the sizes 
M M 

^ and ^of the two sets of normalized deviations to conq>are. 

A weighting function privileges low fiequency range at the expense of high frequency range, to set 

the influence of details in the final result. 

^ wlB denote the dissimilarity fimction and ^ the corresponding similari^ function. 
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a(0 ,0,)=^^! 

M = min(Mi , A/j ) 

E(jc,j')= - — — si x>y 



y 



si y>x 



Jfc=l 

5ww = 1-a(0 ,©2) 



5. Conclusion 

Present document introduced two descr^tors, a shape descriptor for sdQ images in chapter 3 and a shape 
defbimation descrqitor for objects in video sequences in chapter 4 . 

The sb^pe descrq>tor is based on complex Fourier descrqitors whose theory has been explained in 2. 1 . It 
gives a fiequeacial description of the contour of the object. 

First resuhs show that the shape descriptor is both robust and discriminating. It is invariant by translation, 
rotation and scaling and also scalable. It handles resampliog. Tests have even {»oved that downsampling 
increase matching scores. The dedicated matching function allows to set the degree of similarity^ between 
two objects, as explained in 3.4. 

This shape descrqitor is used as a basis to characterize sh^)e deformation in a video sequence and dms to 
define a percentage of variation of each Fourier coefficient. That is possible because of the meaningful 
interpretation of its fiequencial descrqition. 

First results presented inds^e that it is possible to evahiate how mtK:li a shape can be deformed, by looking 
at . Its normalized deviation appears to quantify the degree of deformation. Its value is the deformation rate. 
Even if it is not designed for, this descrq>tor can be consid^ed a signature of the sh^pe deformation and be 
used in a query search to match objects that get out of shape in the same way. 
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CLAIMS : 

1. A descriptor for the representation of a shape of a moving region, said 
descriptor giving a way to characterize a segmented moving object as more or less rigid on the 
basis of input data available in the form of a binary mask of said object, sampled on a regular 
grid. 

2. A descriptor scheme using a shape descriptor according to claim 1. 

3. A video-surveillance device including a description scheme according to claim 2. 

4. A descriptor for the representation of a shape of a moving region, said 
descriptor giving a way to characterize the deformation of a segmented moving object on the 
basis of input data available in the form of a sequence of binary masks corresponding to a 

r 

segmented video sequence corresponding to said object. 

5. A description scheme using a shape deformation scheme according to claim 4, 

6. A video-surveillance device including a description scheme according to claim 5. 
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Abstract 

The invention relates to a representation mode of the shape of a region, 
provided for characterizing a segmented moving object, and to a description scheme using such 
a shape descriptor. 

The invention also relates to a representation mode of the deformation of such 
a shape, provided for characterizing the deformation of an unique moving object within a 
segmented video sequence. 

The invention finally relates to apparatuses such as video-surveillance devices, 
including description schemes that use such shape and shape defomiatlon descriptors. The 
main application of the invention is the future MPEG-7 standard. 
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